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ABSTRACT ' . 

The Comprehensive School Mathematics Program (CSMP) 
IS a program of CEMREL, Inc., one of the national educational 
laboratories, and was funded by the National Institute of Education . 
(NIE). Its major purpose is the development of curriculum materials 
for kindergarten through grade 6. One of the most important questions 
in the evaluation of CSMP is whether the transition to junior high 
school mathematics is made easier or more. difficult because of. pupil 
experience with the program. This study is a preliminary 
investigation, a. comparison of seventh-grade mathematics teachers' 
ratings of former CSMP versus non-CSMP students' performance in 
class. The summary notes the dat^ are far from definitive, but former 
CSMP students seen to be doing at least as well in seventh-grade 
classes as stud.ents from standard programs. Three sites were examined 
where comparison^ between non-CSMP and CSMP pupils was possible. At 
two of these sites, former CSMP pupils received higher teacher 
ratings for participation in class, motivation, creativity and 
problem solving, and practical applications. There were no 
differences between th two groups at a third site. (MP) 
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Description of Eva-luation Report Series 



The Comprehensive School Mathematics Program (CSMP) is a program of.CEMREL, 
Inc. , one of the natix)nal edycationa] Vaboratorie.s i and is funded by the National 
Institute of Education.- Its*major purpose is the development of curriculum 
materials for gpades K-6. , 

' Begin/iing in Sepfgmber^ 1973-, CSMP materials began being used in classrooms • 
^ on a regular basis, beginning in kindergarten and first grade. The evaluation 
activities have paralleled the development- and .dissemi'nati on of materials so that 
the primary evaluation efnphasis is now at the upper elementary 'grades. All 
,. ^gjj^''^^^^ ^^^^ conducted by a group within CEMREL which is independent of 

• The evaluation of the program'in this extended pilot'trial is intended to be 
reasonably comprehensi ve. and to 'supply information xiesired by a wide variety of 
•audiences. For that reason the reports in this series are reasonably non-technical 
and do not attempt to widely explore some of the -related issues. On 'the next page 
^•^ ?ioi?"^ ^^^^ °^ reports through 1980. Below is given a list of reports completed 

Evaluation Report: '.S-B-l Sixth Grade Evaluation, Preliminary Study " " 

8-B-2 Evaluation, of Revised Second Grade, MANS Blue Level 

8-B-3 Evaluation of Revise'd Third Grade, MANS Green Level 

8-B-4 Three Evaluati.ons of Gifted Studerrt Use • . • 

8-C-l Preliminary Study, of CSMP "Graduates" 



ERIC 



m 



Evaluation Report 


1-A-l 


(1974) 


l-A-2 




l-A-3 


• 


l-B-l 




l-B-2 




l-B-3 




1-8-4 




l-B-5 




l-B-6 




1-C-l 


• 


l-C-2 




l-C-3 




l-C-4 




l-C-5 




l-C-6 


Evaluation Report 


2-A-l 


(1975) 


^-B-1 




2-B-2 




2-B-3 




2-C-l 




2-C-2 


• 


2-C-3 


Evaluation Report 


3-B-l 


(1976) 


3-€-l 


Evaluation Report 


4-A-l 


(1977) 


4-B-l 




4-B-2 




4-B-3 




4-C-l 


Evaluation Report 


5-B-l 


(1978) 


5-B-2 




5-C-l 


Evaluation Reoort 


6-B-l 


• (1979) 


6-8-2 




w w i 


Evaluation Report 


7-B^l 


(1980) 


7-B-2 




7-B-3 




7-B-4 




7-B-5 




7-B-6 



Extended Pilot Trials of the 
Comprehensive Schoo? Mathematics Progr'am' 

Evaluation Report Series 

Overview, Design and Instrumentatiorf 
External Review of CSMP Materials 
Final Summary Report Year J 
Mfcl-Year Jest Data: CSMP First Grade Content 
End-of-Year Test Data: CSMP First Grade Content 
End-of-Year Test Data: Standard First' Grade Content 
End-of-Year Test Data: CSMP Kindergarten Content 
Test Data on Some General Cognitive 'Skitls * ' 
• Summary Test Data: Detroit Schools 
Teacher Training Report 
.Observations of , CSMP First Grade Classes 
Mid-Year Data from Teacher' Questionnaires 
End-of-Year Data from Teacher Questionnaires 
interviews with CSMP Kindergarten' Teachers 
Analysis of Teacher Logs 

Final Summary Report Year 2 

Second ^rade Test Data 

Readministration of First Grade Test Items 

Student Interviews " . 

Teacher Questionnaire Data 

Teacher Interviews, Second, Grade ^ / 
Teacher Interviews, First Grade ^ 



Second and Third Grade Test Data Year 3 
Tea\her Questionnaire Data Year 3 



r 



FinanSummary Report Year 4 
Standardized Test Data, Third Grade ' 
Mathematics Applied to Novel Situations (MANS) Test Data 
Individually Administered Problems, Third-Grade 
Teacher Questionnaire Data, Third Grade 

Fourth Grade MANS Test Data * ^ 
Indtvidually Administered Problems, Fourth Grade 
Teacher Questionnaire and Interview Data, Fourth Grade • 

Comparative Test Data: Fourth Grade 
Preliminary Test Data: Fifth Grade 
Teacher Questionnaire Data': Grades 3-5 

Fifth Grade Evaluation: 
Fifth Grade Evaluation: 
Fifth Grade Evaluation: 
Re-evaluation of Second 
Achievement of Former CSMP^ 
^ Student Achievement,' Rapid 



Volume I, Summary 
^Volume II, Test Data 
Volume III, Non-Test Data 
Grade, Revised MANS Tests 
students' at Fourth^ Grade 
Implementation Model 



ERIC 



Key^to Indexing 

Evaluation Reports are labelled m-X-n, 

where m is the year of the pilot study, with 1^73-74 as Year 1. 
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and summaries, B is for student outcomes and C'is for other data, 
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■ , One of the most important questions in the evaluation of CSMP is the 
•ability of CSMP .students to learn and do matlypmatics after they have completed 
the CSMP K-6 program and are enrolled^Jn a "regular" mathematics program. Is the 
transition to junior high school mathematics made easier or more difficult because 
of 't.heir experience with CSMP? 



• Some of the characteristics of CSMP which might affect this transition are 
the following: . ' ' • 

a) Different content such as probability, geometry, , 
- b) Different emphasis (less on computational algorithms, more on 
'various processes), " ' " .• 

c) 'Different instructional strategies for teaching certain topics" 

(such as fractions and decima|s^h^^^^^ • 

d) Longer lessons with frequent teacher-class dialogue (versus 
shorter lessons with frequent individual help'by the teacher), 

e) The SR.iral appro&ch (versus a mastery emphsisis). 



■ The present. study is a preliminary investigation of one aspect of this question, 
a comp'tirison of seventh grade math teachers' ratings of former CSMP versus 
former Non-CsIlP students' performance in class. 
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Design of the Study > 

* « 

So far, there are relatively few districts which have CSMP students who have 
completed six years of CSMP and are now in seventh grade. At three sites, there 

• were at least two classes of "CSMP graduates" wrio were enrolled in the f^l 1 of 1981 
in seventh. grade of a junior high school, which school also contained many seventh 
graders wfio did not study CSMP. This partition into CSMP-Non-CSMP was always based 

• on the elementary school the students attended. 

Table 1 describes these sites. 

V Table/1 

Participating Sites 
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• 




Type of District 


Number of Contributing 
Elementary Schools 


Number of Seventh 
Grade Students 








C5MP 


Non-CSMP . 


CSMP 


Non-CSMP 


• 


Site A 


Large suburban with many junior 
highs. This junior high located 
in farther reaches of district. 
Average SES, mostly white. 


1 


3 * 


48 


118 . 

•> 




Site B 


Inher Suburban with one junior 
high.. Lower middle SES, mostly 
black. 


1 


2* 


-36' 


74* 


• 


Site C 


Suburban with one junior high. 
High SES, mostly white. 


1 


4 


55 


210 



*At Site-B there were 8 Non-CSMP elementary schools altogether. For the purposes'of this study 
only 2 were considered, the other 6 being of quit# di fferent SES and student ability levels. 
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At each junior high school, seventh grade math teachers were asked to rate'^ 
each of their students^ using a five point scale, on the following four characteristics: 

A. Participation in Cla^s 

- Participates frequently^ 

- Shows high-quality of partitipati on 

-*Has good listening skills long attention y)an ^ • 

- Volunteers responses 

B. Moti vati on« - 

- Has strong interest' in subject 

- Is able to work independently > ? 

- Is interested^ in the "why" as well as correctness of answer 

- Is willi.ng to learn new methods and ideas 

C. Creativity & Problem Solving 

" Has good reasoning and logical thinking skills 

- uses new or unusual methods to solve problems 

- Can defend his. ideas 

- Tries more tl^an one way to solve problem ^ 

D. Practical Applications 

- Shows familiarity with conventional tdrms & symbols 

- Is able to organize and interpret information 

- Applies knowledge to practical problems 

- Translates new probl^ins into fami 1 i ar^forms 



• If 
The actual rating form given to teachers is shown in the Appendix. 

Teachers were not told the purpose of the study (but were given a more general 
reason for the assessrnent ) , nor which students were ex-CSMP, .and the foVm made no 
specific references to mathematics and, hence, could, have been viewed by the teacher 
-as a student assessment which could apply to other subject areas as well. These 
ratings were made during the second or third quarters of the school year. 
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At two of the 5ites it was. possible to record for each student, district 
maintained data which could be used as a general measure of abil'ity of the students. 
At Site A, this was the Reading Comprehension score from Iowa Tests of Basic Skills., 
administered in the spring of fifth grade. At Site C, this was the Verbal score 
from the Cognitive Abilities Test, also administered in fifth grade. At Site B, ■ 
no such data was readily availabl£; however the two Non-CSMP elementary schools 
were known from previous evaluation studies to have students with similar ab>fit> 
.levels to the CSMP school. • • ' 

^ 'At each site, a comparison between former CSMP and Non-CSMP students was tl" 
made on each of the four rating ^characteristics . At Sites A and C, an analysis of 
covariance procedure was used to take into account the ability level of students; 
at Site B an analysis of- variance procedure* was used. 

In the section /'Other Results" , seventh grade achievem&nt data from Site C 



grade 'teachers 



is presented. Also given is. a summary of interview data from seventh 
at a fourth site, where all seventh graders were former CSMP students, and hence 
such a comparative study could not be conducted. 



Main Results 

Table 2 sRows the means across students at each site for the ratings and for the 
covariate ability rae^asure, where available. Also given is the number of students 
that this data is based on, i.e:, the rvumber with both a rating and a covariate' 
score. The headings refer to "CSMP and "Non.CSMP"; they were of course Ex-CSMP 
and Ex-Non-CSMP students. * " " . 



Table 2 

Mean Ratings AcrosS' Students 





Site A 


Site B ' 


Site C 




CSMP 


Non- 


■CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


A. Participation 




3. 


4 


3.2 


2.7, 


3.1 


' 2.9 


B. Motivation 


3.2 


3. 


4 


3.4 


2.8 


3.1 ^ 


2.7 


C. Creativity and Problem Solving 


if 

3.1 


3. 


1 


3.1 


2.7 


3.1 


2.7 


D. Practical Applications 


3.0 


3. 


1 


3'.2 


2.7 


3.0 


2.7 


Total of A-D 


12.6 


13. 


0 


12*. 9 


10.8 . 


12.3 


11.0 


Covariate 


41.9^ 


43. 


7^ 






112.3^ 


108.2^ 


Number of Students 


48 


118 




36 


74 


55 


210 " 



Reading -Comprehension, ITBS, spring of 5th grade, rax score. 
Verbal Score, Cognitive Ability Test, 5th grade, raw score. 



These are the mean ratings whi ch dp jiot 'take into account ability levels of 
the students. Ijt can be seen that at Site A, Ex-Non-CSMP students had higher 
scores on the ability njeasure; at. Site C„ Ex-CSMP students scored higher. In 
order to take onto account these differences, an analysis of covariance procedure 
was used for Sites A and C. 



- Table 3, shows the id^usted means for* Sites A and C when ability of student 
is taken into account and* repeats the .observed means for Site B. Also^given is 
the level of significance from .either tbe Analysis- of covariance (A and C) or 
analysis of variance (B),' The degrees of freedom were subst-anti al— at least 
100-- in all cases. ^ ^ 

Table 3 

Adjusted Means (A arid C) or Raw Means (B) 

^ ^ ' 1 
and Significance Tests ' 





Site A 




Site B 




Site C 


• 't 


CSMP 


Non-CSMP 


p-value 


CSMP 


Non-CSMP 


p-value 


CSMP Non-CSMP 


p-value 


A. Participation 


3.3 


3.4 ' 




3.2 


2.7 


.07 


3.1 


3.0 


.50 


B. Motivation 


3.3 


3.3 




' 3.4 


2.8 


.05 


3.0 


2.8 


.30 


C. Creativity and 
' Problem Solving 


3.1 


3.J 




3.1 


2.7 


.11 


3.0 


7 

2.8 


.20 


D. Practical 
Applications 


3.1 


> 3.J 




3.2 


2.7 


.08 


3.0 


2.8 


' .30 


Total of A-D 


12.8 


12.8 




12.9 


10.8 


.05 


12.1 


11.3 


.20 



A dash (^) Indicated not significant at even the .50 level. 



It can be seen that Ex-CSMP s^tudents received significantly or almost ^ 
significantly higher ratings in Site B, non-significantly higher ratings in Site 
C and had virtually identical ratings to Ex-Non-CSMP students in Site A. There 
appears to have been very little discrimination by the teachers among the various 

» 

ratings categories. 
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other Results 

From Site C * ' ■ - 

* 

Site C, a jocal site, had the most complete data. Quarter grades in mathe 
matics were recorded for each student, and a similar analysis of covariance pro 
•cedure was used to assess any CSMP-Non-CSMP difference in future mathematics 
achievement. _ . . • 

Table 4 shows the adjusted mean math grades, by quarter, for Ex-CSMP and 
EX-Non-CSMP students and the p-value of the ANCOVA procedure. ' ' 

Table 4- 

Adjusted Mean Math Grades, 1980-Bl, Site C 
(A = 5, B = 4, etc.) 

V , 





CSMP 


Non-CSMP 


P-Val ue 


First Quartef / 


3.9 


3.6 


.05 


Second Quarjter 


3.8 


3.5 


.02 


Third Quarter 


'3.7 


3.5 


.10 



It can be seen that Ex-CSMP. students had higher math grades in each 
quarter, usually significantly so. 



students were grouped accordiJig to their verbal scoVes, and for eiich .such 
ability range, the mean math grafde was calculated* This data is shown graphically 
in Figure 1 , below. 



Range of Scores 
Cog. Ah. Test^ Verbal 

Nunter of Students 
CSHP/Non-CSMP 




<100 
10/47 



100-109 
16/65 



110-119 
17/52 



^120 
13/41 



Figure -1, Mean quarter math grades by ability level, Site C 
X = Mean for Ex-CSMP group, • = Mean for Ex-Non-CSMP group 



Ex-CSMP students had higher grades at each ability range and, at the (; 
middle ranges, this difference was about half a grade level. 



The same kind of analysis by ability levels of teacher ratings yielded 
the data shown in Figure ^2, below, for the total of the. four ratings. 



Range of Scores, 
Cog. Ab* Test^ Verbal 

Number of Students 
CSMP/Non-CSMP 



Total of Teacher Ratings 




Cog. Ab, Test 



<100 
10/47 



100-109 
16/65 



110-119 
17/52 



130 " (Verbal) 



^120 
13/41 



Figure 2, Mean total of teacher ratings by student ability level. Site C 
X = Mean for Ex-CSMP group, • = Mean for Ex-Non-CSMP group 

. At the lower abtlity ranges, the ratings were about equal;. at the higher 
ranges, Bx-CSMP^ students averaged about 2 points higher. 
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Site D . • 

At a fourth site, all students in grades K-6 u?e CSMP. Hence all of the 
seventh graders (about 140 students), used CSMP and there was no control group 
to formally compare, them with. However, the three seventh grade teachers were, 
interviewed jointly for about an hour, on-site,,' and asked to informally compare ^ 
the present seventh graders (Ex-CSMP) with previous classes of seventh graders 
(Ex-Non-CSMP).- 

The main, and generally agreed upon, reactions of these seventh, grade teachers 
were the^ following: 

• It was expected that these students w6uld hxive poor computation skills; 
in fact the students computed quite w^Jl. " ^ 

• The students^ were particularly vyell behaved; this may have been because 
there was. a ^new English program or because the-seventh grade group just 
happened to" be a "good group." 

\ 

• Students were just as bad as ever in word problems. 

• No mention was made of better problem solving ability or^ thinking skills, 
but at this time of the year (October), not much new work had bgen done— 
mostly review and basics. 

• Students were slightly better listeners, more wllljng to ask questions, 
and more wllling'to vol unteer answers that might be quite wrong. 

• "Seventh graders are seventh graders." ^ 



, 12> , 
i / 



Summary 



The data are far from definitive, but Ex-CSMP students seem to be doing 
at least as well in their seventh grade math classes as Ex-Non-CSMP students 
whfen abili-ty level is controlled, and often they do better. 



At two of three sites. Ex-CSMP students received higher teacher ratings 
for Participation in Class. Motivation. Creativity and Problem Solving, and 
Practical Applications; this advantage' was usually significant in one of -the sites, 
consistent but not quite significant at the other. " 



f 



At the third tite. there were no differences between Ex-CSMP and Ex^Non- 
CSMp students. 

^ Further analysis at one of the sites showed that Ex-CSMP students received 
significantly higher grades in mathematics, especially those in the middle ability 
ranges. • ; 

Interviews with seventh grade math teachers at a fourth site, where CSMP- 
Non-CSMP comparisons were not possible, confirmed the view that CSMP students 
had no difficulty in adapting to the usual classroom activities of seventh grade, 
and were in some ways, better students. 
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APPENDIX 



RATING FORM USED BY SEVENTH GRADE TEACHERS 



' 15 



-Name of Teacher . Please rate each of your students on each of the following characteristics 

Use a 6-point scale: l(low), 2, 3, 4 or 5(high) 



NdMS of StudfKitC 


A. Partkipdtlon In Class 


B. ^totivation 


C. Creativity & Problaq Solvina 


D. Practical Aoplic^finn^ 


- Pdrt ir iiwit^c fr^ntiontlw 

- Shows high quality of 

part ic ipi t ion 

- Has good listening skills 

long attention span 

- Volunteers responses 


nas Strong interest 
In subject 

- Is able to* work 

independently 

- Is interested in the "why** 

*^s wel 1 as correctness 
of answer 
^ 7 Is will ing to learn new 
p^viKwi^ ong iqcis 


- Has good reasoning and 

logical thinking skills 

- Uses new or unusual methods 

to solve problems 

- Can defend his ideas 

- Tries wore than one way to 

sol ve pi'obi em 


- Shows familiarity with con- 

ventional terms & symbols 

- Is able to organize and 

iiivciprct inronnatlon \ 

- Applies knowledge to ^ 

practical problems 

- Translates new problems 

into famil iar forms 


1. 








: ^ — 


2. 










3. ■ 

« 




















5; 










6-.. 

» ■ ■ 








r ' — 


7. 










8. 










9. 


• 








10. 










11. 








J 


12. 










13. 










14. 










15. 














- 






17. 










18. 










19. 










20. 










21. 











Use other side, If necessary 



